Deblocking filters are used in the video coding standards in order to combat blocking artifacts. The blocking artifacts arise because the original video is split into blocks which are processed relatively independently. The blocking artifacts can arise due to different intra predictions of blocks, quantization effects and motion compensation. Two particular variants of deblocking are described below.
1) H.264 Deblocking:
In state of the art video coding, such as H.264, there is an adaptive deblocking filter, also denoted loop filter, after prediction and residual reconstruction, but before storage of the reconstruction for later reference when encoding or decoding subsequent frames. The deblocking filtering consists of several steps, such as filter decisions, filtering operations, a clipping function and changes of pixel values. The decision whether or not to filter a border is made based on evaluation of several conditions. Filter decisions typically depend on macroblock (MB) type, motion vector (MV) difference between neighboring blocks, whether neighboring blocks have coded residuals and on the local structure of the current and/or neighboring blocks.
The amount of filtering for a pixel depends, among others, on the position of that pixel relative to the block border, or block boundary, and on the quantization parameter (QP) value used for residual coding.
The filter decision is based on comparing three pixel differences with three thresholds. The thresholds are adapted to the quantization parameter (QP) value. For instance, assume a vertical block boundary of:a b c d|e f g h where a, b c and d denote the pixel values of the pixels of a row of pixels in a current block with e, f, g and h denoting the corresponding pixel values of the pixels of a corresponding row of pixels in a neighboring block. If the following conditions are fulfilled the filter decision is positive, e.g., abs(d−e)<thr1, abs(c−d)<thr2, and abs(e−f)<thr2, where thr1 and thr2 are adapted based on QP. A horizontal block boundary can be handled in a corresponding way.
There are two filtering modes in H.264. In the first filtering mode, referred to as normal filtering, filtering can be described with a delta value with which filtering changes the current value. The filtering for the pixels closest to the block boundary is:d′=d+delta and e′=e−delta,where delta has been clipped off to a threshold thr3 to a value that is constrained by the QP. More filtering is thereby allowed for high QP than for low QP.
Clipping can be described as:delta_clipped=max(−thr3,min(thr3,delta)), where thr3>=0.where thr3 is controlling the filter strength. A larger value of thr3 means that the filtering is stronger, which means that a stronger low-pass filtering effect will occur.
The filter strength can be increased if any of the following two conditions also holds, e.g., abs(b-d)<thr2 and abs(e-g)<thr2. The filter strength is adapted by clipping the delta less, e.g., by allowing for more variation.
The second filtering mode, referred to as strong filtering, is applied for intra macroblock boundaries only, when the following condition is fulfilled:abs(d−e)<thr1/4.
For more information of deblocking filtering in H.264 reference is made to List et al., Adaptive Deblocking Filter, IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, July 2003.
2) HEVC Deblocking:
As is well-known in the art, a video frame is divided into non-overlapping blocks of pixels that are encoded and decoded according to various available intra and inter coding modes.
Generally, a video frame is divided into non-overlapping macroblocks of 16×16 pixels. Such a macroblock can in turn be divided into smaller blocks of different sizes, such as 4×4 or 8×8 pixels. However, the described method may be applied also to rectangular blocks, arranged e.g., as 4×8, 8×4, 8×16 or 16×8 pixels. The embodiments can be applied to any such block of pixels, including macroblocks or even larger blocks of pixels.
In the emerging High Efficiency Video Coding (HEVC) standard, coding units (CU), prediction units (PU) and transform units (TU) are used. The prediction units are defined inside a coding unit and contain the intra or inter prediction modes. Transform units are defined inside a coding unit where the largest transform size is 32×32 pixels and the smallest size is 4×4 pixels. The CU size is currently varying from 64×64 pixels (largest) to 8×8 pixels (smallest). In this way, the largest CU can be split into smaller CUs with the “level of granularity” depending on the local characteristics of the frame. This means that the largest CU may be split into smaller CUs of different sizes.
Large blocks, referred to as Largest Coding Units (LCU) in HEVC, are scanned left to right in the same way as normal macroblocks in H.264. Each LCU may be split into four smaller coding units (CU), and then may be split again hierarchically in a quad-tree fashion. There is also a smallest size for the Coding Unit defined, these blocks are called Smallest Coding Unit (SCU).
The CU has its prediction type (e.g., intra prediction or inter-prediction). The CU is also a root of two structures called prediction units and transform units. Each prediction unit inside the CU can have its own prediction that is different from the predictions of the other PU (for example, a separate motion vector or intra prediction direction). A CU can contain one PU (which has then the same size as the CU) or can be split further into up to four PUs. Those PUs can have either square or rectangular form (in this case, the vertical and horizontal PU dimensions differ). As an example, there might be a CU of size 16×16 that is split once, creating 4 8×8 prediction unit blocks (PUs). If the coding type of the CU is Intra, the PUs may have different Intra prediction modes. If the coding type of the CU is Inter, the PUs may have different motion vectors.
There is also a transform quad-tree that has the CU as its root. The resulting blocks are called Transform Units (TU). As an example, there might be a CU of size 16×16 that is split into 8×8 TUs. Then, one of the 8×8 TU can be split into 4×4 TUs. Then each TU is transformed with an 8×8 or a 4×4 transform. If the root TU was not split, then a 16×16 transform would have been used. Transforms can also have a non-square (rectangular) shape.
In the draft HEVC (High Efficiency Video Coding) specification “Test Model under Consideration”, ITU-T SG16 WP3 document, JCTVC-B205, Chapter 6.5 In-loop filter process, the deblocking filter works differently from H.264. The filtering is performed if at least one of the blocks on the side of the boundary is intra, or has non-zero coefficients, or if the difference between the motion vector components of the blocks is greater than or equal to one integer pixel. For example, if filtering the border between two blocks with a vertical block boundary of:p3ip2ip1ip0i|q0iq1iq2iq3i with pji denoting the pixel value of pixel number j of row number i in the current block and qji denoting the pixel value of pixel number j of row number i in the neighboring block, i=0 . . . 7, j=0 . . . 3, then the following condition should also be satisfied:d=|p22−2×p12+p02|+|q22−2×q12+q02|+|p25−2×p15+p05|+|q25−2×q15+q05|<βwhere β depends on QP. In the above mentioned HEVC specification, there is a table of β, where β increases with QP.
If the given conditions are fulfilled and filtering is done between a current block and a neighboring block, one of two types of filtering, referred to as weak and strong filtering, respectively, is performed. The choice between the strong and the weak filtering is done separately for each line depending on the following conditions.
For each line i=0 . . . 7, the strong filtering is performed if all the following conditions are true, otherwise, weak filtering is performed:d<(β<<2)(|p3i−p0i|+|q0i−q3i|)<(β>>3)|p0i−q0i|<((5×tC+1)<<1)where tC and β depend on QP and >> denotes a right shift operator.
The two filtering modes, i.e., weak and strong filtering, mentioned in the HEVC specification mentioned above can be described as follows:
Weak Filtering:
Weak filtering is performed based on the conditions specified above. The actual filtering works by computing an offset, adding it to the original pixel value and clipping the sum to a filtered output pixel value in the range of 0-255:offset=Clip(−tC,tC,(13×(q0i−p0i)+4×(q1i−p1i)−5×(q2i−p2i)+16)>>5))p0i=Clip0-255(p0i+offset)q0i=Clip0-255(q0i−offset)p1i=Clip0-255(p1i+offset/2)q1i=Clip0-255(q1i−offset/2)where the clip function Clip(A, B, x) is defined as Clip(A, B, x)=A if x<A, Clip(A, B, x)=B if x>B, Clip(A, B, x)=x if A<=x<=B, and Clip0-255(x) is defined as Clip(0, 255, x).Strong Filtering:
The strong filtering mode is performed by the following set of operations:p0i=Clip0-255((p2i+2×p1i+2×p0i+2×q0i+q1i+4)>>3)q0i=Clip0-255((p1i+2×p0i+2×q0i+2×q1i+q2i+4)>>3)p1i=Clip0-255((p2i+p1i+p0i+q0i+2)>>2)q1i=Clip0-255((p0i+q0i+q1i+q2i+2)>>2)p2i=Clip0-255((2×p3i+3×p2i+p1i+p0i+q0i+4)>>3)q2i=Clip0-255((p0i+q0i+q1i+3×q2i+2×q3i+4)>>3)
In H.264 filtering, there is a check whether or not abs(q0-p0)<threshold(QP) is fulfilled, for the purpose deciding whether there is a blocking artifact at the block boundary or if it is a natural edge at the block boundary. However, such a check is not mentioned in the HEVC standard, probably due to the fact that the deblocking filtering is done also in case the signals remind of inclined ramps. Therefore, such a check cannot tell if one has to do with a natural edge or a part of a change in the ramp signal. As a consequence, natural edges are also filtered with the deblocking filter, although the amount of filtering is limited by a clipping value.
It is therefore a need for an efficient blocking filter that can be used to reduce blocking artifacts at block boundaries but which does not have the above mentioned drawbacks.